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Abstract 


The  present  paper  analyzes  In  detail  (talk-aloud  protocols)  "Good*  and  "Poor" 
students’  Initial  encoding  of  worked-out  examples  of  mechanics  problems,  and  their 
subsequent  reliance  on  examples  during  problem  solving.  We  find  that  "Good"  students 
learn  with  understanding:  they  generate  many  explanations  which  refine  and  expand  the 
conditions  for  the  action  parts  of  the  example  solutions,  and  relate  these  actions  to 
principles  In  the  text.  These  self-explanations  are  guided  by  accurate  monitoring  of  their 
comprehension  failures  and  successes.  Such  learning  results  In  an  example-independent 
knowledge  and  in  a  better  understanding  of  the  principles  presented  In  the  text.  "Poor* 
students  do  not  generate  sufficient  self-explanations.  Inaccurately  monitor  their  learning 
and  subsequently  rely  heavily  on  examples.  The  results  are  discussed  relating  these 
psychological  findings  to  existing  AI  models  of  explanation-based  generalizations. 
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SELF-EXPLANATIONS:  HOW  STUDENTS  STUDY  AND 
USE  EXAMPLES  IN  LEARNING  TO  SOLVE  PROBLEMS 

Learning  Is  a  constructive  process  in  which  the  student  converts  the  words,  and 
examples  from  the  teacher  or  the  text.  Into  usable  skills,  such  as  solving  problems.  This 
process  of  conversion  Is  essentially  a  form  of  constructive  self-instruction  (Simon,  1979). 
Although  the  research  on  the  quality  of  good  teaching  (such  as  those  which  attempt  to 
Identify  the  characteristics  of  a  good  Socratlc  tutor,  Collins  &  Stevens,  1982),  as  well  as 
research  on  the  quality  of  a  good  text  (such  as  those  that  manipulate  the  quality  of 
elaborations,  Reder  &  Anderson,  1980)  may  be  Informative,  ultimately,  learning  rests  on 
the  learning  skills  that  the  students  themselves  bring  to  bear  as  they  learn.  The  goal  of 
this  research  Is  to  understand  the  students'  contribution  to  learning.  To  accomplish  this 
goal,  our  approach  is  to  study  Individual  differences  as  the  students  learn. 

The  majority  of  problem  solving  research  In  the  literature  assumes  that  the 
declarative  knowledge  Is  first  encoded  from  the  text  or  from  the  teacher's  words,  then 
procedurallzed  Into  a  skill.  Most  of  the  literature  has  concentrated  on  the  conversion  of 
already  encoded  knowledge  Into  smooth,  fast,  skillful  problem  solving.  This  conversion 
process  dominates,  for  example.  Anderson's  theory  of  skill  acquisition  (Anderson,  1987). 
In  that  theory,  the  process  of  conversion  Is  achieved  by  using  general  weak  methods 
which  can  convert  declarative  knowledge  Into  domain  specific  procedures  via  the 
mechanism  of  compilation.  Thus.  In  Anderson's  theory.  It  Is  assumed  that  the  effortful 
process  lies  In  the  conversion  of  the  declarative  knowledge  Into  the  procedural 
knowledge,  whereas  the  encoding  of  the  declarative  knowledge  Is  taken  to  be  a 
straightforward  storing: 

In  unanalyzed  form  our  experiences  In  any  domain.  Including  Instruction  (if  it 


Is  available),  models  of  correct  behavior  [worked-out  examples],  successes  and 
failures  of  our  attempts,  and  so  on...  This  means  that  we  can  easily  get 
relevant  knowledge  Into  our  system....  (p.  206). 

We  concur  with  the  common  assumption  that  learning  a  skill  can  be  viewed  as 
encoding  of  Instruction  followed  by  proceduralizatlon  of  some  kind.  However,  our 
research  focuses  on  the  encoding  of  Instruction,  because  a  leading  conjecture  Is  that 
Individual  difference  Is  due  to  the  differences  In  the  encoded  representation  rather  than 
differences  In  conversion  of  the  encoded  instructions  to  skill.  Anderson  (1987)  also 
alluded  to  this  difference  In  the  encoded  representation  as  a  source  of  Individual 
differences  when  he  suggested  that;  "weak  problem-solving  methods  like  analogy  can  be 
much  more  effective  If  they  operate  on  a  rich  representation  of  the  knowledge...*  (p. 
206). 


Our  work  foc  ’sed  on  the  knowledge  that  students  acquire  as  they  read  from  a  text, 
prior  to  their  attempts  to  apply  that  knowledge  In  the  form  of  a  procedural  skill  such  as 
problem  solving.  We  predict  that  learner  differences  arise  primarily  from  the 
representation  that  the  student  has  acquired.  Thus,  we  believe  that  Individual 
differences  do  not  arise  solely  from  the  experiences  the  learner  Is  exposed  to  in  the  form 
of  Instruction  or  text.  Rather,  we  conjecture  that  learner  differences  may  arise  from 
differences  In  the  ways  students  understand  and  learn  from  text,  and  moreover,  these 
learner  differences  subsequently  effect  the  use  of  a  particular  skill  (solving  problems). 

Our  Interest  In  learner  differences  derives  In  part  from  an  attempt  to  understand 
the  acquisition  of  expertise.  While  It  takes  long  periods  of  study  and  practice  before  one 
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can  become  an  expert  In  any  domain.  It  is  clear  that  extensive  practice  and  study  is  a 
necessary  but  not  sufflcient  criterion  for  becoming  an  expert.  Thus,  examinlne  how 
students  differ  in  the  way  they  learn  new  materials  may  shed  light  on  processes  of  skill 
acquisition  that  determine  whether  an  individual  will  or  will  not  achieve  expertise. 

The  part  of  the  text  that  we  focus  on  is  the  worked-out  examples.  This  choice  was 
based  on  theoretical,  empirical,  as  well  as  instructional  reasons.  Theoretically,  there  is 
some  controversy  in  the  literature  concerning  how  generalizations  are  Induced  from 
examples  (see  Dletterlch  &  Michalski,  1983,  for  a  review  of  the  AI  literature,  and 
Murphy  &  Medln,  1985,  for  a  review  of  the  psychological  literature.)  There  are  two 
views.  A  similarity*based  approach  claims  that  generalizations  are  developed  by  inducing 
a  principle  (or  a  set  of  common  features)  from  multiple  examples.  Such  a  principle  would 
embody  the  essential  features  shared  by  all  the  examples.  On  the  other  hand,  an 
explanation-based  approach  to  problem  solving  (Lewis,  1986;  Mitchell,  Keller  &  Kedar- 
Cabelli,  1986)  claims  that  generalization  can  be  obtained  from  a  single  or  a  few 
examples.  Although  many  theories,  including  Anderson’s  ACT*,  can  only  generalize 
from  many  examples,  it  is  clear  that  students  can  often  generalize  from  a  single  example 
(Elio  &  Anderson,  1983:  Kleras  &  Bovalr,  1986).  Providing  evidence  on  how 
generalizations  can  emerge  from  learning  a  few  examples  would  give  credence  to  an 
explanation-based  approach  to  learning. 

Empirically  and  Instructionally,  there  is  a  dilemma  in  the  literature  as  well.  More 
and  more  empirical  evidence  is  emerging  showing  the  importance  of  examples  in 
learning.  Reder,  Charney  and  Morgan  (1986)  for  example,  found  that  the  most  effective 
manuals  for  Instructing  students  bow  to  use  a  personal  computer  are  those  which  contain 
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examples.  LeFevre  and  Dixon  (1986)  found  that  students  actually  prefer  to  use  the 
example  Information  and  Ignored  the  written  Instruction  when  learning  a  procedural 
task.  Plrolll  and  Anderson  (1985)  also  found  that  18  of  their  19  novices  rely  on  analogies 
to  examples  In  the  early  stages  of  learning  to  program  recursion.  Besides  these 
laboratory  findings,  VanLehn  (1986)  also  provided  Indirect  evidence  that  examples  are 
Important  In  regular  classroom  learning.  He  found  that  85%  of  the  systematic  errors  In 
arithmetic,  collected  from  several  thousand  students  could  be  explained  as  deriving  from 
some  type  of  example-driven  learning  process. 

On  the  other  hand,  although  both  students  and  instructional  materials  rely  heavily 
on  worked-out  examples  as  an  Instrument  for  learning,  the  empirical  work  which  directly 
examined  the  role  of  example  solutions  on  problem  solving  found  that  students  who  have 
studied  examples  often  cannot  solve  problems  that  require  a  very  slight  deviation  from 
the  example  solution  (Eylon  &  Helfman,  1982;  Reed.  Dempster.  &  Ettlnger,  1985;  Sweller 
&  Cooper,  1985).  The  discrepancy  between  students’  better  performance  from  text 
materials  that  contain  examples  and  the  students’  failure  to  generalize  from  examples, 
may  be  caused  by  the  degree  to  which  students  understand  the  examples  provided. 
Generally.  In  the  empirical  studies  cited,  no  assessment  Is  made  about  how  well  students 
understood  the  examples.  As  Plrolll  and  Anderson  (1985)  noted,  although  most  of  the 
students  wrote  new  programs  by  analogy  to  example  progr.'ims,  the  success  depended  on 
how  well  the  students  understood  why  the  examples  worked.  Our  work,  in  the  domain 
of  mechanics,  will  shed  light  on  the  degree  to  which  students  understand  an  example  in 
relation  to  their  ability  to  solve  problems. 


The  method  we  use  to  study  how  students  learn  and  understand  an  example  is  via 
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the  explanations  they  give  while  studying  it.  Such  self-explanations  are  Important,  we 
propose,  largely  because  examples  typically  contain  a  sequence  of  actions,  without  much 
explication  of  the  rationale  underlying  the  sequence  of  actions.  Simon  (1979),  for 
example,  noted  that: 

Generally  speaking,  textbooks  are  much  more  explicit  In  enunciating  the  laws 
of  mathematics  or  of  nature  than  In  saying  anything  about  when  these  laws 
may  be  useful  In  solving  problems.  The  actions  of  the  productions  needed  to 
solve  problems  In  the  domain  of  the  textbooks  are  laid  out  systematically,  but 
they  are  not  securely  connected  with  the  conditions  that  should  evoke  them  (p. 

92). 

We  can  provide  a  concrete  example  of  the  Inadequacy  of  worked-out  examples  by 
taking  one  example  from  the  fifth  chapter  of  the  Halllday  and  Res:''  ':  text  (1981),  as 
shown  In  Figure  1.  We  can  see  the  lack  of  specification  of  the  explicit  conditions  under 
which  the  actions  should  be  executed. 

For  example.  It  Is  not  clear  in  Statement  2  of  this  example-exercise  why  one  should 
■consider  the  knot  at  the  Junction  of  the  three  strings  to  be  the  body."  This  Is  a  critical 
piece  of  Information  because  It  Implies  that  at  this  location  (as  opposed  to  the  block),  the 
sum  of  the  forces  Is  zero.  Such  lack  of  specification  of  the  explicit  conditions  for  actions 
occurs  throughout  the  example.  In  Statement  0,  how  doer  the  student  know  that  F^,  Fg, 
and  F^  are  all  the  forces  acting  on  the  body,  and  that  there  are  no  others?  Statement  7 
Is  essentially  a  restatement  of  Newton’s  First  Law,*  but  It  requires  chaining  several 


inferences  together,  and  translating  them  Into  an  equation  (for  example,  because  the 
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30 


45 


Figura  S-6  Exampla  S.  (a)  A  block  of  waignt  W  is  susoenoeo 
by  strings,  /b)  A  frae-Ooby  diagram  sboming  all  tne  forces 
acting  on  tna  knot.  The  strings  are  assumed  to  be  weigntless. 


1.  Figure  S-8t  shows  an  object  of  weight  W  hung  by  massless  strings. 

2.  Consider  the  knot  at  the  junction  of  the  three  strings  to  be  "the  body*. 

3.  The  body  remains  at  rest  under  the  action  of  the  three  forces  shown 

Fig.  5-«. 

4.  Suppose  we  are  given  the  magnitude  of  one  of  these  forces. 

5.  How  can  we  find  the  magnitude  of  the  other  forcesf 

*•  F.i  F’oi  ‘f*  »«‘ing  00  fhe  body. 

A  o  ^ 

7.  Since  the  body  is  unaccelerated.  ^ b  *  ** 

8.  Choosing  the  x-  and  y-axes  as  shown,  we  can  write  this  vector  equation 

as  three  scalar  equations: 


8. 

to. 


f'Ay-^'^By-^'^Cy-O 
tl.  using  Eq.  S-2.  The  third  scalar  equation  for  the  z-axis  is  simply: 


12. 


As 


•^Bs-^C,-® 


13.  That  is.  the  vectors  all  lie  in  the  x-y  plane  so  that  they  have  no  z 


components. 

14.  From  the  figure  we  see  that 

15.  F^  —  -F^  cos  30  —  -0.88«F^. 

o 

18.  F.  —F.  sin  30  —  O.SOOF.. 
Ay  .A  A 

17.  and 

e 

18.  Fg^j  Fg  COS  45  —  O.rOTFg. 

e 

18.  Fgy  *  Fg  sin  45  «  0.707  Fg. 


Figure  1;  A  siriDES  example,  taken  directly  from  Halllday  and  Resnlck 

(IffSl) 
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body  Is  at  rest,  there  are  no  external  forces,  therefore  the  sum  of  the  forces  on  the  body 
must  equal  to  zero).  Statement  8  Is  totally  unexplained:  why  are  the  axes  chosen  as 
such?  It  Is  clear  that  the  solution  steps  within  an  example  are  not  explicit  about  the 
conditions  under  which  the  actions  apply. 

In  order  to  learn  with  understanding,  a  student  needs  to  overcome  the 
incompleteness  of  an  example  by  drawing  conclusions  and  making  inferences  from  the 
presented  information  (VVlckelgren,  1974).  To  do  so,  a  student  needs  to  provide 
explanations  for  why  a  particular  action  Is  taken.  Only  then  will  the  student  be  able  to 
apply  the  acquired  procedure  to  non-lsomprphlc  problems  that  do  not  match  exactly  the 
conditions  of  the  example  solution.  Thus,  we  suggest  that  a  good  student  'understands* 
an  example  solution  and  will  succeed  In  generalizing  because  he/she  makes  a  conscious 
effort  to  explain  and  extrapolate  the  principle,  and  to  ascertain  the  conditions  of 
application  of  the  solution  steps  beyond  what  Is  explicitly  stated.  Consequently,  we  ask 
In  this  paper  how  students  can  come  to  understand  an  example. 

It  Is  difficult  to  define  what  'understanding*  means  In  the  context  of  learning  from 
examples.  Operationally,  our  study  was  designed  to  assess  understanding  with  three 
measures:  solution  of  Isomorphic  problems,  solution  of  far  transfer  problems,  and 

elaborations  generated  during  studying  examples.  The  weakest  method  to  detect 
understanding  Is  to  observe  how  successfully  students  solve  very  similar  problems,  since 
very  similar  problems  can  often  be  solved  by  a  simple  syntactic  mapping  of  the  example 
procedure  to  the  to-be-solved  problem.  Another  method  Is  seeing  If  students  can 
successfully  use  the  principle  Involved  In  the  example  In  a  different  and  complex 


problem,  (far  transfer),  because  such  problems  prevent  students  from  being  able  to  solve 


them  via  a  syntactic  mapping.  However,  to  a  certain  extent,  studying  far  transfer  reveals 
only  that  understanding  exists  and  allows  one  to  see  what  conditions  facilitate  It. 

The  method  which  permits  the  most  direct  assessment  of  understanding  of  an 
example  Is  to  examine  the  explicit  elaborations  that  students  provide  while  studying  It. 
Elaborating  Is  a  mechanism  of  study  that  allows  students  to  explain,  Infer,  and  explicate 
the  conditions  and  consequences  of  each  procedural  step  In  the  example.  We  postulate 
that  elaborations  can  reveal  students'  understanding  by  showing  whether  or  not  they 
know  (a)  the  conditions  of  application  of  the  actions,  (b)  the  consequences  of  actions,  (c) 
the  relationship  of  actions  to  goals,  and  (d)  the  relationship  of  goals  and  actions  to 
natural  laws  and  to  other  principles.  This  paper  focuses  primarily  on  the  elaboration 
results. 

In  the  actual  procedure  used,  we  invited  students  to  explain  to  themselves  what 
they  understand  after  every  line  of  a  worked-out  example.  Often  they  made  no 
comments  at  all.  However,  when  they  did  generate  a  comment,  this  method  allowed  us 
to  yoke  the  explanation  to  the  statement  line  In  the  example  so  that  we  could  Interpret 
their  protocols  more  easily. 

Method 

Subjects 

Eight  students  (3  males  and  5  females)  were  selected  from  responses  to  a  campus 
advertisement.  Six  of  them  were  working  towards  bachelor's  degrees  with  varying 
majors.  Two  of  the  8  students  had  additional  post-graduate  training  In  psycholog>'.  None 
of  the  students  had  a  college  physics  course,  although  all  of  them  (except  one)  had  taken 


high  school  physics,  with  differential  performance  (reported  grade)  in  that  course.  We 


Intentionally  chose  students  with  a  range  of  abilities  In  terms  of  grade-point  average 


that  we  could  examine  learner  differences.  Students  were  paid  for  their  participation. 

Procedure  and  Materials 

In  the  present  paper  we  focus  on  how  students  study  three  worked-out  examples  of 
problems  dealing  with  the  application  of  Newton’s  laws  of  motion  and  on  how  this  Initial 
learning  relates  to  their  subsequent  problem  solving.  However,  since  the  relevant 
chapter  demands  substantive  background  subject  matter,  the  study  of  examples  and  the 
problem  solving  tasks  are  embedded  within  a  longitudinal  study  In  which  students  In  our 
laboratory  studied  Newtonian  Mechanics.  Students  learned  the  new  material  In  a  way 
they  would  normally  do  when  studying  on  their  own.  In  terms  of  the  amount  of  time 
devoted  to  studying,  the  rate  of  self-pacing,  and  the  use  of  studying  habits,  such  as 
highlighting  significant  parts  of  the  text  or  rereading.  The  laboratory  learning  differed 
from  the  way  students  would  typically  learn  on  their  own  In  that  all  the  learning  took 
place  In  the  laboratory,  and  that  students  gave  talk-aloud  protocols  while  studying 
examples  and  solving  problems.  The  students  spent  between  8  to  29  hours  to  complete 
the  study,  spread  over  several  weeks.  Figure  2  presents  a  diagram  summarizing  the 
experimental  procedure  for  the  whole  study.  Basically,  the  study  consisted  of  2  major 
phases:  knowledge  acquisition  and  problem  solving. 

Knowledge  Acquisition 

During  the  first  part  of  the  knowledge-acquisition  phase,  subjc  ts  studied  the 
necessary  background  subject-matter,  covering  the  topics  of  measurement,  vectors,  and 
motion  In  one  dimension.  These  materials  are  covered  In  the  first  throe  chapters  of 
Halllday  &  Resnlck.  The  fourth  chapter  on  .Motion  In  a  Plane  was  eliminated  because  It 


did  not  have  direct  bearing  on  learning  Chapter  5.  the  target  chapter  on  Particle 
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Figure  2:  A  dlaeram  depicting  the  design  of  the  study 
Dynamics.  Decisions  of  this  kind,  as  well  as  designing  of  questions  and  problems,  were 
made  after  consultations  with  physicists. 


i  I 

For  each  of  the  three  background  chapters,  students  read  through  and  studied  the 
chapter.  They  were  requested  to  record  on  a  separate  sheet  of  paper  any  questions 
which  arose  during  their  study.  Each  chapter  also  contained  a  note  to  stop  and  get  the 
experimenter  in  order  to  give  a  verbal  protocol  of  one  worked-out  example.  This  protocol 
was  meant  mainly  as  practice  in  giving  protocols.  When  the  students  believed  they  were 
ready  to  be  tested  on  the  material  in  the  chapter,  they  notified  the  experimenter,  and 
were  then  required  to  produce  correct  answers  to  a  set  of  Declarative,  Qualitative  and 
Quantitative  questions,  given  in  that  order.  Declarative  questions  were  designed  to 
assess  the  recall  of  critical  facts  from  each  chapter,  e.g.,  *What  is  the  difference  between 
a  scalar  and  a  vector?"  Some  of  these  questions  came  from  the  textbook  and  some  were 
designed  by  the  experimenters.  The  Qualitative  questions  were  designed  to  assess 
reasoning  and  Inferences  about  the  concepts  in  each  of  the  chapters  without  reference  to 
quantities,  e.g.,  "Can  an  object  have  an  eastward  velocity  while  experiencing  a  westward 
acceleration?"  The  majority  of  these  problems  were  taken  from  the  textbook,  several 
were  generated  by  the  experimenters.  The  Quantitative  questions  assessed  procedural 
skills  for  quantitative  problem  solving,  e.g.,  "Two  bodies  begin  free  fall  from  rest  at  the 
same  height,  i.O  sec  apart.  How  long  after  the  first  body  begins  to  fall  will  the  two 
bodies  be  10  meters  apart?"  All  of  the  Quantitative  problems  came  from  those  presented 
in  the  text  at  the  end  of  the  chapters. 

After  solving  each  problem  set,  students  returned  the  problems  to  the  experimenter 
for  grading.  If  no  errors  were  made,  they  proceeded  to  the  next  set  of  questions.  If  an 
error  was  made,  the  questions  were  returned  for  correction.  The  Incorrect  answers  were 
identified  and  students  were  given  references  to  those  sections  of  the  text  which 
addressed  lliese  questions.  For  Incorrect  Quantitative  questions  the  correct  final  answer 
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was  also  provided,  analogous  to  the  common  practice  of  looking  up  the  answer  In  the 
back  of  the  book.  Students  then  attempted  to  correct  the  answers  and  resubmitted  them 
for  grading.  If  there  were  still  errors  on  the  second  try,  students  were  provided  with  a 
worked-out  solution  for  the  incorrect  answers.  Students  then  studied  the  worked  out 
solution,  and  after  it  had  been  removed,  were  asked  to  change  their  answers  and/or  to 
reproduce  the  correct  solution  from  memory.  If  after  these  two  tries  the  answer  was  still 
Incorrect,  the  experimenter  explained  the  worked-out  solution  and  the  student  had  to 
reproduce  it.  Thus,  the  first  three  chapters,  consisting  of  the  background  subject  matter, 
were  studied  until  students  could  correctly  solve  a  set  of  Declarative,  Qualitative  and 
Quantitative  problems  relevant  to  each  chapter. 

During  the  second  part  of  the  knowledge  acquisition  phase,  students  studied  tlie 
target  chapter  on  particle  dynamics  (Chapter  5  of  Halllday  &  Resnlck).  Studying  the 
target  chapter  proceeded  exactly  as  for  the  other  chapters,  except  that  students  were 
required  to  answer  correctly  the  relevant  Declarative  and  Qualitative  questions  before 
studying  the  three  worked-out  examples.  This  was  done  to  assure  that  the  students  had 
acquired  the  relevant  declarative  knowledge  from  the  text. 

The  study  of  examples  was  the  major  focus  of  the  knowledge  acquisition  phase; 
students  studied  three  worked-out  examples,  taken  directly  from  the  text.  (See  Figure  1 
for  an  example  of  one  of  the  worked-out  solutions.)  Each  example  solution  represented  a 
■type"  of  problem.  There  were  three  types:  a  strings,  an  Inclined  plane,  and  a  pulley 
problem.  (See  Figure  3.) 


Students  talked  out-loud  while  studying  the  examples,  and  their  protocols  were  taped. 
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Problem  solving 

During  the  problem  solving  phase,  students  solved  two  main  sets  of  problems.  The 
first  -ct  consisted  of  12  "Isomorphic  problems",  with  four  problems  corresponding  to 
each  "type"  of  example  (see  the  shaded  areas  In  the  problem  solving  phase  of  Figure  2). 
These  problems  were  specially  designed  by  us  so  that  they  varied  In  their  degrees  of 
similarity  to  the  worked-out  examples  studied  in  the  previous  phase.  Figure  4  shows  one 
set  of  Isomorphs  corresponding  to  the  "strings"  example. 

The  second  set  consisted  of  seven  "chapter"  problems,  which  were  problems  taken 
directly  from  the  end  of  the  target  chapter.  In  terms  of  our  criteria  for  Isomorphism  to 
example  problems,  these  problems  can  be  considered  "far  transfer"  problems.  Students 
solved  both  sets  of  problems  while  giving  talk-aloud  protocols,  and  no  feedback  was 
provided.  Only  the  problem  solving  protocols  of  the  first  set  of  three  Isomorph  problems 
will  be  analyzed  In  detail  for  this  paper. 

Results 

As  Indicated,  this  paper  will  report  on  the  results  of  the  elaboration  protocols  while 
students  study  the  example  solutions,  presented  In  the  target  chapter,  as  well  as  how 
they  use  the  examples  In  the  problem  solving  protocols  of  the  Isomorphic  problems. 
Individual  differences  will  be  reported  by  contrasting  the  performance  of  "good"  and 
"poor"  students.  These  two  groups  were  defined  post  hoc,  using  a  median  split  on  the 
problem  solving  successes  of  our  students  on  the  12  iso.^iorphlc  and  the  7  chapter 
problems.  In  scoring  the  problems,  arithmetic  slips  were  not  considered  as  errors  and 
partial  solutions  were  credited  proportionally.  All  problems  received  the  same  weight,  so 
that  the  maximum  score  for  the  isomorphic  problems  was  12  and  the  maximum  score  for 
the  chapter  problems  was  7.  (All  scoring.  Identification  and  classification  of  protocols 
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A  block  b  huclnt  from  three  etrlacs-  U  the  teaslon  In  strlnt  l  b  18  N,  whnt  b  the  mass 
of  the  block?  y  X  y' y'y  ^  y  '  y" 

< Wiwq  ^  ^  — r^*****^ 

y  X  - - - -  4 

Similar  1 - - 


rope  A  be?  -«-n«».t»-.jTiTri'.-ar^  cv 

\  TO*  60/ 


Changed  Gravity 


A  balloon  b  belnc  held  down  by  three  massless  ropes.  If  the  balloon  b  pullint  up  with  a 
force  of  300  N.  what  would  the  tension  In  rope  A  be? 


Changed  Foree  Direction 


Three  forces  are  boldlnt  a  800  ks  block  motionless  on  a  frrc'.ionless  surface.  If  force  A  b 
■SO  N.  wbat  would  force  C  be? 


New  Surface 


Figure  4;  The  set  of  four  Isomorphs  to  the  strings  example 


were  performed  by  two  Judges,  with  a  mean  Interrater  reliability  across  all  the  analysis 
ranging  between  80%  and  94%. ) 


There  were  four  students  In  each  group.  The  mean  success  of  the  Good  students 


was  82%  (96%  for  the  Isomorphic  problems  and  68%  for  the  chapter  problems).  The 
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mean  success  of  the  Poor  students  was  46%  (62%  for  the  Isomorphic  and  30%  for  the 
chapter  problems). 

Learning  from  Examples:  Elaboration  protocols 
Amount  of  Elaborations  Generated:  Unit  of  Analyses. 

Our  first  analysis  began  with  a  very  gross  count  of  the  amount  of  elaborations 
students  provided  while  studying  the  example  solutions.  An  elaboration  Is  a  generic  term 
we  use  to  Indicate  any  statement  made  that  Is  not  a  first  reading  of  an  example  line,  nor 
Is  It  any  conversation  carried  on  with  the  experimenter  that  does  not  refer  to  the  subject 
matter  of  the  example,  such  as  "Do  you  have  a  calculator  that  I  can  use?*,  nor  Is  It  a 
response  to  the  experimenter's  requests  to  speak  louder.  Figure  5  Is  a  transcript  of  one 
student's  elaboration  protocol.  During  the  transcription,  segmentation  was  made  roughly 
at  pause  boundaries.  We  also  assigned  a  line  number  to  each  line  of  the  protocols. 
Counting  each  line  that  Is  not  a  reading  or  experimenter's  comment  as  a  line,  this 
example  shows  a  total  of  13  lines.  Notice  that  because  we  are  not  using  sentence 
boundary  as  a  line  boundary  necessarily,  a  short  comment  such  as  "Okay*  Is  counted  as 
one  line,  since  It  Is  Inserted  between  different  activities,  such  as  reading. 


Table  1  shows  the  average  number  of  lines  generated  per  example  by  the  Good  and 
Poor  students,  as  well  as  the  average  amount  of  lime  It  took  them  to  study  an  example. 
These  differences  are  contrasted  with  the  average  number  of  protocol  lines  generated  and 
the  amount  of  time  taken  to  solve  an  Isomorphic  problem!  The  Good  students  generated 
a  considerably  greater  number  of  elaboration  lines  than  the  Poor  students  (142  lines  vs. 
21  lines,  1(6)  =  1.97,  p  <  0.05).  These  elaboration  activities  necessarily  led  the  Good 
students  to  spend  more  time  on  each  example  as  well  (13  min.  vs.  7.4  min.,  t(6)  =  2.16. 


P  <  .05). 
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Read  Line  6: 

Fa,  Fh,  and  Fc  are  all  the  forces 
acting  on  the  body,  (pause) 

(I) 

Elaboration: 

Monitoring 

1)  Okay. 

Read  Line  7. 

Since  the  ...  (pause) 

Experimenter: 

(Okay,  what  are  you  thinking  about?) 

(II) 

Elaboration: 

Monitoring 

2)  I’m  trying  to  think  where  Forces  Fb  and  Fa 

3)  are  going  to  get  the  thing. 

(III) 

Elaboration: 

Explanation 

4)  They’d  Just  be  the  force,  the  rest  mass  of 

5)  the  thing  holding  It  up  would  be  the  force. 

6)  It  could,  well,  actually  It’d  be  the  force  of 

weight. 

(IV) 

Elaboration: 

Explanation 

7)  Cause  being  upheld  by. ..It's  the  resistance 

8)  to  weight  W. 

(V) 

Elaboration: 

Explanation 

9)  It  would  all  be  equal. 

10)  The  change  In  the  uh,  negative  Y  will  the.. 

(VI) 

Elaboration: 

Explanation 

11)  The  force  of  negative  Y  will  be  equal  to 

12)  the  force  in  the  positive  Y. 

(VII) 

Elaboration: 

Explanation 

13)  And  they'll  all  equal  out. 

Figure  5:  An  example  of  a  transcript  of  example-studying  protocols  (SlOl) 


We  rule  out  the  possibility  that  Good  students  are  simply  more  articulate  or  fluent, 
thus  producing  more  elaborations,  since.  In  contrast,  the  number  of  lines  they  generate 
while  solving  problems  Is  approximately  the  same  as  the  Poor  students  (141  vs.  122  lines 
per  problem,  see  Table  l,  bottom).  This  suggests  that  the  students  generate  as  many 
lines  of  protocols  as  they  deem  necessary  In  order  to  learn  the  example.  Likewise,  the 


Table  1 


Amount  of  Elaboration  Generated  While 
Studying  Examples  and  Solving  Problems 


Example- 

Studying 

Protocols 

No.  of  Lines 

No.  of  Min. 

GOOD 

142 

13 

POOR 

21 

7 

No.  of  Statements 
or  Single  Ideas 

51 

18 

Isomorphic 

No.  of  Lines 

141 

122 

Problem 

Solving 

No.  of  Min. 

13.8 

14.3 

Protocols 

Note.  The  numbers  represent  averages  per  student  per  example. 

fact  that  the  Good  students  took  more  time  studying  each  example  solution  than  the 
Poor  students  (13  min.  vs.  7.4  min.)  also  reflects  their  choice  to  spend  more  time  on  each 
example,  rather  than  a  tendency  to  dwell  unnecessarily  on  the  examples,  since  the 
amount  of  time  they  spent  solving  each  Isomorphic  problem  Is  about  the  same  as  the 
time  spent  by  the  Poor  students  (13.8  min.  vs.  14.3  min.). 
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The  assumption  we  make,  that  elaborations  are  generated  If  some  knowledge  or 
Inference  Is  being  procei^sed  or  constructed  In  memory,  Is  no  differ ^nt  than  assumptions 
made  with  reference  to  other  types  of  dependent  measures.  For  example,  the  assumption 
underlying  protocol  analyses  Is  that  the  problem  solver  Is  saying  what  he  Is  thinking 
about  or  dumping  the  content  of  his/her  working  memory.  Thus,  longer  protocols  simply 
refer  to  a  greater  degree  of  processing.  Similarly,  in  research  using  eye  movement 
protocols,  the  assumption  Is  that  the  student  is  processing  the  location  at  which  he/she  is 
fixating  (Just  &  Carpenter,  1976).  Thus,  longer  fixations  Imply  that  the  student  Is 
spending  more  time  processing  that  location.  Therefore,  we  view  the  greater  amount  of 
elaboration  produced  by  the  Good  students  as  a  natural  consequence  of  wanting  to 
understand  the  solution  example  better,  rather  than  the  possibility  that  they  are  more 
articulate  and  fluent. 

It  might  be  claimed  that  the  reason  that  Good  students  learned  more  Is  Just  that 
they  spent  more  time  studying.  However,  this  Is  not  a  very  deep  explanation,  but  merely 
a  restatement  of  the  correlation.  The  Important  question  Is  what  the  Good  students  did 
while  they  were  studying.  As  will  be  seen  shortly,  there  are  such  large  qualitative 
differences  In  what  students  did  while  studying  examples  that  the  simple,  shallow, 
explanation— that  studying  twice  as  long  makes  one  learn  twice  as  much— Is  quite 
implausible  because  It  cannot  explain  the  other  differences  In  students’  learning  behavior. 

However,  although  we  claim  that  longer  protocols  do  not  necessarily  imply 
verbosity,  using  lines  as  a  unit  of  analysis  does  not  seem  adequate  for  capturing  what  Is 
going  on.  That  Is,  an  analysis  at  the  level  of  lines  Is  too  fine-grained  to  characterize  the 
nature  of  these  comments.  Instead,  many  lines  seem  to  refer  to  a  single  Idea,  and  thus 
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we  further  collapsed  the  lines  into  units  expressing  a  single  idea.  For  example,  the  13 
protocol  lines  In  Fig.  5  have  been  recoded  Into  7  elaboration  'statements"  (I-VTI).  Thus, 
basically  the  boundaries  were  placed  on  the  basis  of  different  ideas,  pauses,  or  different 
sort  of  activities  including  reading  or  experimenter’s  Interjections. 

Using  this  sort  of  parsing,  we  see  In  Table  l  that  Good  students’  protocols  have 
been  reduced  from  142  lines  to  51  statements  and  Poor  students’  protocols  remain 
roughly  the  same  (that  is,  18  statements  vs.  21  lines).  Thus,  in  a  sense  we  are  penalizing 
the  Good  students  for  being  articulate.  What  the  data  actually  shows  is  that  the  Good 
students  often  elaborate  quite  a  bit  on  a  single  idea  either  because  they  realize  that  they 
do  not  understand  It  or  they  want  to  provide  a  more  complete  explanation  (see  next 
sections),  or  the  Ideas  are  Just  harder  to  explain  succinctly.  Lewis  and  Mack  (1982)  have 
also  noticed  that  learners  often  spontaneously  offer  explanations  of  why  things  happened 
the  way  they  did.  Even  with  this  unit  of  analysis,  the  Good  students  elaborate 
significantly  more  frequently  than  the  Poor  students  (51  vs.  18.  t(6)  =  1.98,  p  <  .05). 
Henceforth,  all  analyses  will  be  based  on  the  number  of  single  Ideas  or  statements. 

Kinds  of  Elaborations  Generated. 

Elaboration  statements  can  be  classified  Into  three  types.  (See  Fig.  6  for  a 
breakdown  of  all  the  classifications  to  be  discussed.) 

An  elaboration  is  considered  to  be  an  explanation  If  It  says  anything  substantive  about 
the  physics  discussed  In  the  example  statement.  The  following  comment  would  be 
considered  an  explanation: 


’Ummm,  this  would  make  sense,  because  since  they’re  connected 
by  a  string  that  doesn 't  stretch.  * 
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Elaborations 


Failures  Elaborations 


Reline/  Explicate/  Impose  Give  meaning 
expand  infer  a  goal  to  quantitative 

corrditions  consequences  expressions 


Figure  0;  A  classification  scheme  Illustrating  be  decomposition  of 

the  analyses  of  the  protocols. 

or 

'If  the  string's  going  to  be  stretched,  the  earth  s  going  to  be  moved, 
and  the  surface  of  the  incline  is  going  to  be  depressed. ' 

Other  examples  can  be  seen  in  Figure  5. 

An  elaboration  Is  considered  a  monitoring  statement  If  it  refers  to  states  of 
comprehension.  For  example,  remarks  such  as: 


22 


*/  can  see  now  how  they  did  it  * 
or 

was  having  trouble  with  F-mgsin9  =  0* 

would  be  considered  monitoring  statements. 

A  third  category  Includes  several  other  types  of  statements,  mostly  paraphrasing, 
mathematical  elaborations,  and  metastrateglc  statements.  Paraphrasing  would  be 
comments  that  restate  what  the  example  line  said.  For  example,  a  student  who  read  the 
example  line 

Notice  that  the  magnitude  of  T  is  always  intermediate 
between  the  weight  of  mj  and  the  weight  of 

remarked  that 

’Tension  is  always  intermediate  between  the  two...  ’. 

Such  a  statement  would  be  coded  as  a  paraphrase.  Another  example  would  be  the 
remark 

'There  are  no  more  forces ' 
after  reading  the  example  statement 

Fa,  Fb,  and  Fc  are  all  the  forces  acting  on 
the  body. 

A  mathematical  elaboration  would  be  a  statement  such  as 
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’If  this  is  the  sine  of  ^5  degrees,  then  this  is  the  cosine 
of  ^5  degrees 

.\ad  a  metastrateglc  statement  concerns  what  the  students  are  doing  or  planning  to  do, 
such  as; 


’I'm  going  to  look  at  the  diagram’ 
’I'm  going  to  reread  it’ 

’This  is  something  to  remember’. 


Table  2  shows  the  distribution  of  the  three  categories  of  statements  for  the  Good 
and  Poor  students. 


Looking  at  Table  2,  one  can  see  that  although  the  Good  students  produced  a 
significantly  greater  total  number  of  elaborations  (50.9)  than  the  Poor  students  (17.5), 
they  do  not  differ  in  the  distribution  of  the  proportion  of  each  type  of  elaboration  they 
provide.  This  similarity  in  the  distribution  suggests  that  the  difference  between  the  Good 
and  Poor  students  may  Just  be  a  quantitative  one.  so  that  a  simple  remedial  treatment 
for  the  Poor  students  is  simply  to  ask  them  to  spend  more  time  and  effort  at  producing 
elaborations.  However,  closer  examination  of  each  of  the  categories  suggests  that  the 
difference  is  not  a  straightforward  quantitative  one.  Below,  we  analyze  the  differences  in 
the  structure  and  content  of  the  Explanations,  as  well  as  differences  in  the  Monitoring 
Statements. 

Analyses  of  Explanations. 

Of  all  the  elaboration  statements  produced  by  the  Good  students,  31*^  are 
explanations  relating  to  the  physics  content  as  compared  to  2-4‘^  for  the  Poor  students. 
Even  though  both  groups  produced  proportionately  a  similar  amount  of  explanations,  the 
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Table  2 


Types  of  Elaborations 


GOOD 

POOR 

Prop. 

Freq.* 

Prop. 

Freq.* 

Physics 

Explanation 

31% 

15.9 

24% 

4.3 

Monitoring 

Statements 

39% 

20.1 

42% 

7.3 

Others 

(Include 

Paraphrase) 

30% 

14.9 

34% 

5.9 

Total 

100% 

50.9 

100% 

17.5 

♦An  average  number  of  statements  per  student  per  example. 

absolute  number  of  these  explanations  Is  significantly  different  between  the  two  groups 
(15.9  per  example  vs.  4.3,  F(l,6)  =  7.25.  p  <.05).  Also,  the  correlation  between  the 
number  of  physics  explanations  and  the  subsequent  success  In  solving  the  Isomorphic 
problems  Is  very  high  (r  =  .76,  p  <  .05).  Thus  It  seems  that  these  explanations  play  an 
important  role  In  learning  from  examples. 

Structure  of  the  Explanations.  One  way  to  analyze  the  explanations  students 
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provide  upon  studylns  example  statements  is  to  analyze  the  structure  of  their 
expianations.  by  which  we  mean  the  form  or  purpose  of  what  they  say.  Since  our 
conjecture  in  this  research  was  that  students  often  fall  to  understand  example  solutions 
because  the  example  provides  neither  a  clear  specification  for  why  each  procedural  step 
Is  taken  nor  an  explication  of  the  consequences  of  each  step,  this  would  Imply  that  If 
students  do  understand  an  example  solution  well,  they  themselves  must  explicate  the 
conditions  and  consequences  of  each  solution  step.  Hence,  we  attempted  to  capture  this 
characterization  in  the  choice  of  the  analysis  cateeorles.  Analyzins  the  physics 
explanation  statements,  we  could  classify  them  Into  two  major  categories:  Justifications 
and  diagrammatic  statements  (see  Fig.  6  again).  Diagrammatic  statements  are 
statements  that  basically  describe  in  words  what  was  depicted  in  the  diagram.  For 
example,  a  statement  made  by  a  student  in  reference  to  the  diagram  depicted  in  Figure  i 
was: 

*Okay,  so  three  forces  are  on  the  two  strings 
and  from  the  string  going  down  to  the  object.  * 


Justifications  on  the  other  hand,  are  explanations  that  either: 

1)  Refine  or  expand  the  conditions  of  ^  action.  For  example,  in  response 
to  Line  IS  of  the  Inclined  plane  example,  which  stated  that 

It  is  convenient  to  choose  the  x-sxis  of  our  reference  frame 
to  be  along  the  incline  -and  the  y>axis  to  be  normal  to  the 
incline 

student  Si  10  explained  the  conditions  of  such  a  choice  by  saying; 

*and  it  is  very,  umm,  wise  to  choose  a  reference  frame  that's 
parallel  to  the  incline,  parallel  and  normal  to  the  incline,  because 
that  way,  you  'll  have  to  split  up  mg,  the  other  forces  are  already, 
component  vectors  for  you.  ’ 
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2)  Explicate  or  Infer  additional  Implications  of  M  action.  In  response 
to  the  same  line  In  the  Inclined  plane  example,  student  SP2  would  say: 

*So  we  can  save  1  force.. .we  save  the  F  force.  Wie  save 
it  so,  that  I  don ’t  have  to  calculate  it  by,  with  angles... 

So  we  can  save  two  forces.  Partitioning  of  two  forces.  * 

3)  Impose  a  goal  or  purpose  for  an  action.  In  response 
to  the  same  Line  18.  again.  Si  10  said: 

’Basically  it  looks  like  they  are  going  to  split  up  these  three 
forces  into  their  respective  components.  * 

4)  Give  meaning  to  a  set  of  quantitative  expressions.  In  response  to 
Lines  13-16  of  Fig.  1,  which  simply  states  two  equations, 
student  Si  10  stated: 

’Vmm,  and  looking  along,  they've  done  the  same  thing  for  FsubB. 
They’ve  separated  it  into  its  components,  into  its  component  vectors, 
and  they  are  basically  able  to  figure  out  umm, 

the  tension  in  each  of  the  strings  and  how  much  W  weights  providing 
that...  * 

Another  student  (SPi)  responded  similarly  by  saying: 

*i\ow  they  are  going  to  do  the  same  thing 
with  it  to  the  Y’. 


These  latter  two  quotes  show  that  not  only  can  students  understand  in  a  sense  the 
meaning  and  purpose  of  the  quantitative  expressions  (which  Is  a  procedure  for 
decomposing  the  forces),  but  they  are  also  basically  providing  a  goal  for  the  set  of 
actions.  They  refer  to  the  goal  of  the  equations  as  the  "same  thing*. 


Table  3  shows  the  distribution  of  the  two  kinds  of  Explanation  statements.  .\s  can 
be  seen,  almost  all  of  the  Good  students’  explanations  (98%  or  15.5  statements)  are  of 
the  Justification  kind,  whereas  a  fairly  high  proportion  (359o)  of  the  Poor  students’ 
explanations  are  diagrammatic  statements.  (The  Interaction  Is  significant  at  the  .01 
level,  F(l,6)  =  20.55.)  The  characteristic  of  the  diagrammatic  statements  Is  that  they  do 
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not  necessarily  ad«l  any  new  Information.  In  some  sense,  they  are  paraphrasing  Into 
words  what  Is  shown  plctorlally,  whereas  the  Justifications  are  explanations  of  various 
sorts  as  discussed  above.  Diagrammatic  statements  were  classified  as  "explanations* 
rather  than  Included  In  the  "other"  category  since  they  were  not  paraphrases  of  textual 
statements. 

Table  3 

Kinds  of  Explanation 


GOOD 

POOR 

Prop. 

Freo." 

Prop. 

Freq." 

Justifications 

98% 

15.5 

65% 

2.75 

Diagrammatic 

Statements 

2%, 

0.3 

35% 

1.5 

Total 

100% 

15.8 

100% 

4.25 

"An  average  number  of  statements  per  student  per  example. 

Content  of  the  Explanations.  So  far,  we  have  just  given  a  characterization  of  the 
structure  of  the  explanation  statements.  A  majority  of  the  explanations  justify  actions 
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Stated  In  the  text;  they  refine  or  expand  the  conditions  of  an  action,  explicate  the 
consequences  of  an  action,  provide  a  goal  for  a  set  of  actions,  and  explain  the  meaning  of 
a  set  of  quantitative  expressions.  However,  there  is  an  important  difference  in  the  Good 
and  Poor  students’  explanations  that  is  not  easily  captured  by  assessing  the  structure  of 
what  they  say.  This  difference  requires  an  analysis  of  the  content  of  the  explanations. 

We  can  first  give  a  descriptive  analysis  of  the  difference  between  the  Good  and 
Poor  students'  justifications  by  contrasting  the  following  explanations  generated  after 
reading  the  equation  from  the  Pulley  example: 

T  -  mjg  =  nija 


A  poor  student  (S109)  comments: 

*OK,  cause  the  acceleration  is  due  to  gravity.  * 

This  comment,  at  best.  Is  an  Incomplete  statement.  It  does  not  capture  the  interrelation 
between  the  force  of  tension  and  the  force  due  to  gravity.  On  the  other  hand,  a  Good 
student  (SPl),  In  response  to  the  same  example  line  remarks: 

*OK,  so  its  basically  a  way  of  adding  them  together  and  seeing  if  there  is 
anything  left  over.  And  if  there  is  anything  left  over,  it  equals  the  force: 
mass  times  acceleration. ' 

Such  a  comment  Is  not  only  more  complete,  but  It  shows  that  the  Good  student  Is  trying 
to  understand  the  example  by  relating  the  example  statements  to  explanations  and 
principles  stated  In  the  text,  and  understanding  how  one  example  statement  follows  from 


the  previous  one(s).  This  Is  typical  of  Good  students’  explanations. 


29 


Our  descriptive  analyses  of  the  nature  and  content  of  the  explanations  so  far 
indicate  that  the  Good  students  seem  to  generate  explanations  which  relate  to  the 
principles  stated  In  the  text,  as  well  as  relating  the  consecutive  example  statements  to 
each  other  (as  when  they  Impose  a  goal  on  a  set  of  actions,  or  give  meaning  to  a  set  of 
quantitative  expressions).  VVe  can  further  capture  the  degree  to  which  students' 
explanations  reflect  the  principles  learned  from  the  text  by  Judging  the  extent  to  which 
explanations  can  be  said  to  be  guided  by  these  principles.  For  example,  the  following 
response  to  Line  3  of  Figure  1: 

*So  that  means  that  they  have  to  cancel  out,  only  the  body  tuouldn  7  be  at 
rest.  * 

would  be  Judged  to  be  guided  by  Newton's  First  Law  that  If  there  Is  no  motion,  then  the 
sum  of  the  forces  must  be  zero.  Thus,  to  Increase  the  rigor  of  our  analyses,  we  further 
restricted  our  analyses  to  only  those  explanations  that  are  derived  or  Inferred  from 
.Newton's  Three  Laws,  About  a  third  of  the  Good  students'  explanations  (4.75  out  of 
15.8  statements)  could  be  derived  from  or  referred  to  Newton's  Laws,  whereas  nearly 
none  of  the  Poor  students'  explanations  (0.7  out  of  4.25  statements)  related  to  any  of 
Newton's  Laws. 

One  might  think  that  more  of  the  Good  students'  explanations  are  derivable  from 
the  principles  In  the  text  than  the  Poor  students'  explanations  because  the  Good  students 
have  understood  the  principles  better  prior  to  studying  the  examples.  This  Interpretation 
can  be  ruled  out  by  examining  the  content  of  the  students'  responses  to  the  Declarative 
questions  administered  prior  to  the  studying  of  the  example-exercises.  Recall  that  the 
students  had  to  be  able  to  answer  a  set  of  Declarative  questions  to  the  satisfaction  of  the 
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experimenters  before  they  could  proceed  to  the  example-studyinc  phase  of  the  study. 
One  part  of  this  declarative  test  asks  the  students  to  state  In  their  own  words  Newton's 
Three  Laws.  We  analyzed  those  responses  using:  the  same  analysis  used  In  the  Chi,  Glaser 
and  Rees  (1982)  Study  Six.  In  this  analysis  each  of  the  Newton’s  Laws  was  decomposed 
Into  several  subcomponents.  For  example.  Newton's  Second  Law,  F=ma.  has  been 
decomposed  Into  four  subcomponents; 

1.  Applies  to  one  body 

2.  Involves  all  forces  on  the  body 

3.  Net  force  Is  the  vector  sum  of  all  the  forces 

4.  F=ma,  or  the  mas:nltude  of  F  Is  ma.  and  the  direction  of  a  Is  the  same  as  F. 

Using  these  components  as  a  scoring  criterion.  It  can  be  seen  In  Table  4  that  for  the 
answers  to  the  Declarative  questions,  both  the  Good  and  the  Poor  students  encoded  5.5 
components  (out  of  a  total  of  12  possible  components  for  the  three  Laws).  Thus,  before 
studying  the  examples,  the  two  groups  do  not  differ  In  the  amount  of  declarative 
knowledge  that  they  have  encoded. 


In  order  to  compare  the  student's  Initial  understanding  of  the  Three  Laws  with  their 
subsequent  understanding  as  manifested  in  the  explanations,  we  used  the  same 
component  analysis  to  code  those  explanations  which  were  previously  Judged  to  be 
derivable  from  Newton's  Three  laws.  Within  these  subsets  of  explanations,  the  Good 
students  verbalized  5.5  components  of  the  Three  Laws  whereas  the  Poor  students  only 
stated  1.25  components  (t(6)  =  2.14,  p<.05). 


31 


Table  4 

Number  of  Components  of  Newton’s  Three  Laws  (Out  of  12) 


GOOD 

POOR 

Declarative  Test 

5.5 

5.5 

Expianations 

5.5 

1.25 

New  Compon'-nts 

3.0 

0.25 

Note,  The  numbers  represent  averages  per  student. 

What  Is  most  remarkable  Is  that  for  the  Good  students.  3  out  of  the  5.5 
components  stated  In  the  explanations  are  distinct  from  those  previously  mentioned  In 
the  answers  to  Declarative  questions.  In  other  words,  while  example-studying,  the  Good 
students  gained  3  additional  components  of  the  Three  Laws.  For  example,  three  out  of 
four  Good  students  inferred  or  understood  the  second  component,  that  the  force 
■Involves  ail  the  forces  on  the  body.*  This  particular  component  of  the  Second  Law  was 
not  mentioned  Initially  by  any  of  the  Good  students  In  the  answers  to  the  Declarative 
questions.  However,  self-explanations  produced  while  studying  the  examples  obviously 
Instantiated  this  point.  This  suggests  that  the  Good  students  can  learn  from  the 
examples,  perhaps  because  the  examples  Instantiate  components  of  the  Laws  that  were 
not  particularly  salient  from  reading  a  declarative  description  of  It  In  the  text.  Perhaps 


^  . 
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also,  this  Is  why  the  Good  students  feel  so  compelled  to  "explain"  while  studying,  since 
they  are  learning  and  encoding  new  knowledge. 

The  Poor  students.  In  contrast,  hardly  added  any  new  components  to  what  they 
already  knew  (0.25  new  components,  see  Table  4).  The  difference  in  the  number  of  new 
components  added  Is  significantly  different  between  the  Good  and  the  Poor  students  (t(6) 
=  3.67,  p<.01).  None  of  the  Poor  students  articulated,  for  example,  the  second 
component  of  Newton's  Second  Law,  In  either  their  explanations  or  their  declarative 
answers.  Thus,  the  Poor  students  did  not  add  much  new  knowledge  to  what  they 
already  knew  about  the  Three  Laws,  and  further,  did  not  use  what  they  did  know. 

Summary.  Our  analyses  of  explanations  show  that  not  only  do  the  Good  students 
produce  a  significantly  greater  number  of  explanations,  but  their  explanations  consist 
almost  entirely  of  Justifications  which  offer  Inferences  about  the  conditions,  the 
consequences,  the  goals,  and  the  meaning  of  various  mathematical  actions  described  In 
the  example.  On  the  other  hand,  over  a  third  of  the  Poor  students'  explanations  are 
diagrammatic  statements  which  do  not  add  any  Inferences  to  what  Is  already  described. 
Furthermore,  the  explanations  that  the  Good  students  provide  are  largely  guided  by  the 
principles,  concepts,  and  definitions  Introduced  In  the  text. 

We  found  that  both  the  Good  and  the  Poor  students  had  basically  the  same 
understanding  of  these  Three  Laws  Initially.  In  terms  of  the  number  of  components  of 
the  Three  Laws  that  they  understood,  as  cited  In  their  answers  to  Declarative  questions. 
(.According  to  Greeno  &  Riley,  1987,  explicit  verbalization  Is  the  strictest  criterion  for 
assessing  understanding  of  a  principle).  The  self-explanations  provided  by  the  Good 
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students  while  studying  the  examples  made  their  understanding  of  the  principles  more 
complete  (8.5  out  of  12  components).  For  example,  after  their  self-explanations,  the 
Good  students  understood  3  out  of  the  4  components  of  Newton’s  Second  Law  (the  only 
component  missing  Is  the  first  one),  whereas  prior  to  that,  they  only  understood  one  of 
the  four  components  (the  last  one).  In  contrast,  since  the  Poor  students  did  not  provide 
many  explanations,  they  did  not  advance  their  understanding  of  the  Laws  after  studying 
the  examples  (5.75  out  of  12  components).  For  example,  for  the  Second  Law,  no  new 
component  was  added  to  what  they  already  knew  (which  was  only  the  fourth  component 
of  the  Second  Law).  Hence,  their  understanding  of  the  Second  Law  after  studying  the 
example  was  Just  as  Incomplete  as  It  was  before  studying  the  example.  In  fact,  they  did 
not  use  as  many  components  In  their  explanations  as  they  could  have  from  their  Initial 
understanding. 

Since  all  the  students  seem  to  have  encoded  the  same  number  of  components  of  the 
Three  Laws  Initially,  we  are  assured  that  the  students  did  acquire  the  same  amount  of 
relevant  knowledge  from  the  chapter.  In  particular,  the  Good  students  did  not  have  a 
better  understanding  of  Newton’s  Laws  prior  to  studying  the  examples.  However,  we 
further  propose  that  by  the  time  students  attempt  to  solve  problems,  their 
representations  of  the  principles  and  other  declarative  knowledge  Introduced  In  the  text 
will  differ  depending  on  the  degree  to  which  their  understanding  of  the  principles  Is 
enhanced  during  their  studying  of  examples.  Such  an  Interpretation  Is  consistent  with 
the  findings  showing  that  students  prefer  text  materials  that  contain  examples  (Reder  et 
al.,  1986;  LeFevre  &  Dixon,  1986).  Thus,  even  though  both  the  Good  and  the  Poor 
students  can  begin  problem  solving  by  applying  general  weak  methods  (the  process  of 
procedurallzatlon,  .\nderson,  1987),  the  Good  students  will  achieve  greater  success  In 
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solving  problems  since  an  Instantiation  of  the  weak  methods  depends  on  the 
representations  of  their  declarative  knowledge,  which  Is  more  complete.  Thus,  example- 
studying  Is  a  critical  phase  of  skill  acquisition. 

It  Is  Interesting  to  compare  these  findings  to  our  expert-novice  results.  In  which  the 
experts  cited  many  more  components  In  their  summaries  of  Newton’s  Three  Laws  than 
the  novices  (Chi,  et.al.,  1982).  In  contrast,  before  studying  the  examples,  our  Good  and 
Poor  students  did  not  differ  In  their  declarative  understanding.  Both  groups  have 
relatively  poor  understanding.  As  a  result  of  differential  learning  from  examples,  the  two 
groups  developed  knowledge  differences  which  are  compatible  with  differences  we  found 
between  experts  and  novices.  This  suggests  that  experts  became  that  way  not  because 
they  were  better  at  encoding  the  declarative  knowledge,  but  their  greater  understanding 
was  probably  gained  from  the  way  they  studied  examples  and  solved  problems. 

While  we  do  not  understand  the  mechanism  underlying  self-explanations  at  this 
point.  It  Is  clear  that  self-explanations  not  only  construct  better  problem  solving 
procedures,  but  they  also  help  the  students  understand  the  underlying  principles  more 
completely.  (More  speculations  about  the  mechanisms  of  self-explanations  will  be 
presented  In  the  Discussion). 

Monitoring  Statements 

So  far.  we  have  found  that  the  Good  students  tend  to  explain  an  example  to 
themselves  (more  so  than  the  Poor  students),  and  that  these  explanations  are 
manifestations  of  learning  that  the  Good  students  are  undertaking  while  studying.  The 
Poor  students  are  not  learning,  at  least  not  learning  the  components  of  the  Three  Laws, 
while  studying  the  examples.  Another  possibility  for  why  the  Poor  students  are  not 
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elaborating  Is  that  they  may  not  realize  that  they  do  not  understand  the  material; 
another  possibility  Is  that  they  may  actually  think  that  they  do  understand  the  material, 
thus  they  need  not  elaborate.  To  see  If  these  conjectures  are  viable,  we  have  examined 
the  kind  of  elaborations  that  have  been  categorized  as  monitoring  statements.  As  we 
noted  earlier  in  Table  2.  around  40%  of  the  elaboration  statements  are  of  this  kind  (30% 
for  the  Good  students  and  42%  for  the  Poor  students). 

Monitoring  statements  can  be  further  broken  down  Into  those  that  Indicate  that 
the  student  understood  what  was  presented  In  the  example  line  (such  as  •Okay,"  or  "I 
can  see  now  how  they  did  It")  and  those  that  Indicate  that  the  student  failed  to 
understand  (usually  questions  raised  about  the  example  line,  such  as  ’Why  is  mg3in& 
negative?’).  By  examining  the  proportion  of  each  type  of  monitoring  statements,  (see 
table  5),  it  seems  that  the  Poor  students  detect  comprehension  failures  less  frequently 
than  the  Good  students.  Thus,  the  Poor  students,  within  each  example,  generated  an 
average  of  only  l.l  statements  Indicating  that  they  failed  to  comprehend  a  line;  whereas 
the  Good  students  generated  an  average  of  9.3  such  comments  (t(6)  =  2.32,  p<.05). 
Common  sense  would  have  predicted  that  the  Poor  students  should  have  detected 
comprehension  failures  more  frequently,  rather  than  less  frequently,  since  they  were  less 
successful  at  solving  problems  than  the  Good  students. 

Why  Is  It  Important  to  be  able  to  detect  comprehension  failures?  We  surmise  that 
It  Is  Important  to  be  able  to  detect  comprehension  failures  In  order  for  students  to  know 
that  they  ought  to  do  something  to  try  and  understand.  One  way  to  assess  this  Is  to 
look  at  the  frequency  with  which  a  detection  of  comprehension  failure  Is  followed  by 
explanations.  Indeed,  for  both  the  Good  and  Poor  students,  detections  of  comprehension 
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Table  6 

Monitoring  Statements 


GOOD 

POOR 

Prop. 

Freq.* 

Prop. 

Freq.* 

Comprehension 

Failure 

46% 

9.3 

15% 

1.1 

Understand 

53% 

10.8 

85% 

6.2 

Total 

100% 

20.1 

100% 

7.3 

•.\n  average  number  of  statements  per  student  per  example. 


failures  do  Initiate  explanations,  although  more  so  for  the  Good  than  for  the  Poor 
students,  both  proportionately  and  on  an  absolute  basis.  Elghty-flve  percent  (or  8.0  out 
of  9.3  statements)  of  the  Good  students'  and  60%  (or  0.6  out  of  l.l  statements)  of  the 
Poor  students’  detection  of  comprehension  failure  Is  followed  by  explanation,  (F(2,12)  = 
12.5,  p  <  .01).  These  results  suggest  that  one  crucial  advantage  of  the  Good  students  Is 
In  their  ability  to  spontaneously  Identify  the  loci  of  their  comprehension  failures,  which 
In  turn  Initiate  the  necessary  Inferenclng  process. 


Not  only  do  the  Poor  students  often  fall  to  realize  that  they  do  not  understand,  but 
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on  the  few  occasions  that  they  do  think  they  do  not  understand,  these  occur  strictly  at 
loci  of  quantitative  expressions,  whereas  only  half  of  the  Good  students'  detection  of 
comprehension  failure  arise  from  such  loci.  The  other  half  reside  In  places  where  we 
believe  Important  physics  principles  and  concepts  are  being  explained.  Our  favorite 
example  Is  the  third  line  In  Fig.  1, 

Consider  the  knot  at  the  junction  of  the  three 
strings  to  be  the  body. 

At  this  location,  3  of  the  4  Good  students  Indicated  comprehension  failure,  whereas  none 
of  the  Poor  students  did.  (This  Is  a  critical  piece  of  information  because  It  tells  the 
student  that  the  knot  and  not  the  mass  of  the  block,  should  be  the  center  of  the 
reference  frame  at  which  all  the  forces  have  to  sum  to  zero.) 

There  Is  also  a  qualitative  difference  In  the  kind  of  questions  that  detection  of 
comprehension  failure  raises.  For  example.  Poor  students,  when  detecting  their  failure  to 
comprehend,  often  state  their  lack  of  understanding  In  a  general  way,  such  as: 

'We//,  what  should  you  do  here?’ 

or  restate  the  equation  that  they  do  not  understand,  such  as: 

'/  was  having  trouble  with  F-mgsind=0’. 

On  the  other  hand,  the  questions  that  the  Good  students  raise  are  specific  Inquiries  about 
the  physical  situation  described  In  the  example.  For  example, 

’I’m  wondering  whether  there  would  be  acceleration  due  to  gravity?’ 
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or 


’Why  the  force  has  to  change?’ 

Thus,  It  appears  that  the  specific  questions  that  the  Good  students  pose  can  be  answered 
by  engaging  in  self-explanations,  since  the  processes  of  self-explanations  are  essentially 
the  processes  of  Inferring  the  conditions  and  consequences  of  actions,  inducing  goals,  and 
so  on.  On  the  other  hand,  the  general  lack  of  understanding  as  posed  by  the  questions  of 
the  Poor  students  cannot  be  resolved  by  engaging  In  self-explanations. 

In  summary,  the  analyses  of  the  monitoring  statements  essentially  show  that  the 
Good  students  realize  that  they  do  not  understand  more  often  than  the  Poor  students. 
The  Poor  students.  In  fact,  seldom  detect  comprehension  failures.  When  they  do.  It 
always  occurs  at  loci  of  quantitative  expressions.  The  ability  to  detect  comprehension 
failures  is  Important  because  such  states  (of  Incomprehension)  tend  to  initiate 
explanation.  When  the  Poor  students  do  detect  comprehension  failures,  their  sense  of  It 
Is  vague  and  general,  whereas  Good  students  ask  very  specific  questions  about  what  they 
don’t  understand.  These  specific  questions  can  potentially  be  resolved  by  engaging  In 
self-explanation.  In  short,  the  Poor  students  seem  oblivious  to  the  fact  that  they  do  not 
understand.  In  part  because  they  only  have  a  superficial  understanding  of  what  they 
read.  In  the  next  section,  we  will  discuss  how  these  different  patterns  of  learning  dictate 
how  examples  are  used  during  problem  solving. 

Problem  Solving  Protocols  and  References  to  Examples 
We  now  turn  to  our  analyses  of  student’s  problem  solving  protocols  with  reference 
to  the  use  of  examples  for  the  12  Isomorphic  problems  (4  per  each  of  the  3  example 
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problems  in  the  text).  A  global  analysis  Is  to  examine  the  extent  to  which  Good  and  Poor 
problem  solvers  use  examples.  References  to  examples  were  Identified  In  three  ways:  (a) 
the  student  could  make  explicit  remarks  such  as,  *Now,  I'm  going  to  look  a*  the 
example.*;  (b)  the  student  could  be  looking  at  the  example,  rereading  some  of  the 
example  lines;  and  (c)  the  student  could  respond  that  she/he  is  looking  at  the  example  if 
the  experimenter  probed  and  asked  what  she/he  was  doing. 

Both  the  Good  and  Poor  students  use  examples  frequently.  The  Good  students 
referred  to  the  examples  In  0  out  of  the  12  Isomorphic  problems,  and  the  Poor  students 
referred  to  examples  In  solving  10  out  of  the  12  problems.  Thus,  at  this  global  level,  of 
whether  or  not  students  reference  examples,  our  finding  is  consistent  with  those  of  PlrolU 
and  Anderson  (1085),  as  well  as  with  the  common  intuition  that  students  do  refer  to 
examples  in  learning  to  solve  problems.  However,  a  more  detailed  analysis  shows  that 
they  reference  examples  in  a  very  different  way. 

Reference  to  Example  Episodes 

At  a  more  detailed  level,  we  analyzed  the  number  of  'episodes*  in  which  students 
refer  to  examples  within  each  problem  solving  protocol.  Only  the  first  set  of  three 
isomorphic  problems  (one  per  example)  are  used  for  this  analysis.  We  analyzed  only  the 
first  set  because  we  were  Interested  In  the  students'  Initial  use  and  reliance  on  the 
studied  example;  subsequent  problem  solving  could  have  benefited  from  using  procedures 
that  were  learned  while  solving  the  first  set. 

We  can  classify  example  usage  Into  three  categories:  (a)  Reading,  (b)  Copy  and 
Map,  and  (c)  Compare  and  Check.  A  Read  episode  Is  simply  when  students  reread 
verbatim  one  or  more  example  lines.  A  reading  episode  Is  marked  by  a  reference  to  the 
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example,  reading  some  consecutive  lines  (with  or  without  comments  Inserted),  and 
terminating  when  the  student  goes  back  to  solving  the  problem.  Copy  and  Map^  episodes 
are  instances  where  the  student  copies  from  the  example  either  an  equation,  labels  from 
the  diagram,  the  free  body  diagram  Itself,  or  the  axes.  The  following  quote  for  student 
Si 05  would  be  coded  as  mapping: 

*OK,  so  choosing  the  axis  from  the  diagram,  it  would  be  better  to  tilt  it  SO 
degrees.  * 

Compare  and  Check  episodes  are  those  In  which  the  student  turned  to  the  example  to 
check  a  specific  subprocedure  or  a  result,  either  before  or  after  an  Independent  attempt 
on  the  solution.  For  example,  one  of  the  students  (SPl)  forgot  what  the  units  for  weight 
were  after  resolving  two  vectors  Into  four  components  and  referred  back  to  the  example 
with  the  comment 

*/n  the  example  problem,  how  did  they  refer  to  weight?*. 

Table  6  shows  the  mean  number  of  episodes  of  each  kind  for  the  problem  solving 
protocols  of  the  first  set  of  Isomorphic  problems.  Although  we  showed  earlier  that  at  a 
global  level,  both  the  Good  and  Poor  students  use  examples  equally  often,  this  more 
detailed  analysis  shows  that  within  a  problem  solving  protocol,  the  Good  students  do 
refer  less  often  to  examples  than  the  Poor  students  (2.7  episodes  per  problem  vs.  6.8 
episodes  per  problem),  although  the  difference  Is  only  marginally  significant  (F(l,6)  = 
3.46,  p  <  .10).  (It  Is  difficult  to  obtain  dramatic  differences  statistically  because  the 
grain-size  of  this  particular  unit  of  analysls-the  number  of  eplsodes-ls  rather  large.) 
The  less  frequent  references  to  the  example  suggest  that  the  Good  students  probably 
have  extracted  more  out  uf  the  examples  while  studying  It  (as  evidenced  by  the  greater 
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amount  of  elaboration),  so  that  they  may  not  need  to  refer  to  it  as  often  during  problem 
solving. 

This  interpretation  Is  supported  further  by  comparing  the  different  kinds  of 
reference  to  example  episodes.  The  distribution  of  the  different  episodes  is  signlHcantly 
different  for  the  two  groups  (F(3,18)  =  3.81,  p  <  .05  for  the  Interaction),  predominantly 
because  there  Is  a  substantial  difference  in  the  number  of  Reading  episodes  between  the 
Good  and  Poor  students  (0.6  vs.  4.1,  HSD  p  <  .05,  Tukey  test).  That  Is,  consistent  with 
our  interpretation,  because  the  Poor  students  did  not  get  much  out  of  the  examples  when 
they  were  studying  It.  they  now  need  to  re-read  them.  (See  Table  6).  There  are  no 
differences  In  the  number  of  Map  and  Check  episodes. 

We  further  suggest  that  the  Poor  students  need  to  reread  the  examples  not  only  because 
they  did  not  get  much  out  of  It  while  studying,  but  that  they  are  using  the  examples  to 
■find  a  solution*.  This  can  be  seen  by  the  number  o'  lines  that  students  read  within 
each  Reading  episode.  The  Poor  students  read,  on  average,  a  larger  number  of  example 
lines  per  episode  than  the  Good  students  (13.0  vs.  1.6  lines,  F(3.18)  =  11. 1.  p  <  .01). 
This  difference  also  suggests  that  the  function  of  the  rereading  for  the  Good  and  Poor 
students  Is  different.  The  Good  students  seem  to  reread  one  line  in  the  example  as  a  way 
to  locate  certain  relevant  Information  they  need  In  order  to  check  and  compare  their 
solutions,  whereas  the  Poor  students  reread  several  lines  until  they  encounter  an 
equation  that  they  can  map.  Once  they  come  to  the  relevant  information,  the  Good  and 
Poor  students  use  the  same  number  of  lines  for  mapping  (1.3  and  1.5  lines  respectively) 
and  for  checking  (0.5  and  0.3  respectively). 
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Table  6 


Use  of  Example  during  Problem  Solving 


No.  of  Episodes 

Per  Problem 

GOOD 

POOR 

Read 

0.8 

4.1 

Map 

1.1 

2.1 

Check 

1.0 

0.2 

Total 

2.7 

6.6 

Within  Each  Read 

Episode: 

No.  of  Lines 

Read: 

1.6 

13.0 

Loci  of 

Entry: 

Equation 
or  FED 

first  line 

Note.  The  numbers  represent  averages  per  student  per  problem. 


The  fact  that  Good  students  reread  a  specific  line  In  the  example  while  solving  a 
problem  suggests  that  they  consult  the  examples  after  they  have  a  plan  or  formulated  an 
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idea  on  bow  to  solve  the  problem,  whereas  the  Poor  students  use  the  examples  as  a  way 
to  find  a  solution  that  can  be  copied.  Another  way  to  support  this  conjecture  Is  to  look 
at  the  location  at  which  students  begin  reading  the  example  lines.  All  the  Poor  students, 
for  their  first  encounters  with  problem  solving  (during  the  first  isomorphic  problem), 
started  rereading  the  example  from  the  very  first  line;  whereas  none  of  the  Good 
students  started  reading  from  the  beginning.  The  Good  students'  first  Interactions  with 
an  example  usually  consisted  of  referring  to  an  equation  or  to  a  free-body  diagram. 

The  contrast  In  our  Interpretation  can  also  be  substantiated  by  the  goals  the 
students  state  while  referring  to  the  examples.  Good  students  usually  enter  examples 
with  a  very  specific  goal.  One  student  (SlOl),  for  example,  said  while  referring  to  an 
example  line: 

*/’m  looking  at  the  formula  here,  trying  to  see  how  you  solve  for  one  (Force 
1)  given  the  angle  *, 

whereas  Poor  student  refer  to  examples  with  a  general  global  goal,  such  as: 

*VV?ia<  do  they  do?' 

then  proceed  to  read  the  first  line. 

Learning  From  an  Example  During  Problem  Solving 

Since  the  Poor  students  spend  a  considerable,  amount  of  effort  re-reading  the 
examples  while  solving  problems,  it  may  be  that  some  students  prefer  to  learn  In  this 
context,  while  others  prefer  to  study  without  a  specific  problem  solving  goal.  To 
evaluate  this  conjecture,  we  looked  for  explanations  generated  during  reference  to 
example  episodes  within  the  problem  solving  protocols  of  the  first  set  of  Isomorphic 
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problems  for  each  of  the  three  examples.  Table  7  shows  the  number  of  each  kind  of 
explanations,  using  the  same  taxonomy  we  used  earlier  (see  Table  3). 

Table  7 


Frequency  of  Explanations  Generated  While 
Referring  to  Examples  in  the  Context  of 
Problem  solving 


GOOD* 

POOR 

Justification 

2.0 

2.5 

Diagrammatic 

Statement 

0.25 

0.25 

Total 

2.25 

2.75 

Note.  The  numbers  represent  averages  per  student  per  problem. 


♦Only  one  Good  student  provided  explanations. 


First  of  all.  even  though  the  Good  students  had  as  many  explanations  as  the  Poor 
students,  (2.25  vs.  2.75),  all  the  explanations  of  the  Good  students  were  generated  by  one 
student  (SI  10)  who  had  generated  the  least  amount  of  explanations  while  studying 
examples.  Many  of  the  explanations  provided  by  the  Good  students  (as  shown  In  Table 
3)  were  generated  by  only  three  out  of  the  four  Good  students.  Hence,  this  particular 
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Good  student  (Si  10)  does  fit  our  conjecture  that  some  students  prefer  to  explain  while 
studylnc  and  others  prefer  to  explain  while  solving  problems.  However,  the  amount  of 
explanations  provided  by  the  Poor  students  In  the  problem  solving  context  Is  nowhere 
near  the  level  that  the  Good  students  generated  while  studying  examples  (Compare 
Table  7  with  Table  3).  If  we  combine  the  results  of  the  explanation  data  generated 
during  problem  solving  together  with  those  generated  during  studying,  (l.e..  Tables  3  and 
7),  the  resulting  table  will  have  exactly  the  same  pattern  as  Table  3. 

Discussion 

Our  research  queried  the  extent  to  which  Individual  difference  In  learning  to  solve 
problems  Is  attributable  to  difference  in  the  way  knowledge  Is  encoded  from  the 
exampleexerclses.  We  found.  In  general,  that  Good  students  (those  who  have  greater 
success  at  solving  problems)  tend  to  study  example-exercises  In  a  text  by  explaining  and 
providing  Justifications  for  each  action  (l.e.,  their  explanations  refine  and  expand  the 
conditions  of  an  action,  explicate  the  consequences  of  an  action,  provide  a  goal  for  a  set 
of  actions,  relate  the  consequences  of  one  action  to  another,  and  explain  the  meaning  of  a 
set  of  quantitative  expressions).  More  Importantly,  their  explanations  relate  the  actions 
to  the  principles  and  concepts  In  the  text,  which  In  turn  further  enhance  their 
understanding  of  the  principles.  Essentially,  Good  students  read  the  example  with 
understanding.  Poor  students  do  not  often  explain  the  example-exercLses  to  themselves. 
When  they  do,  their  explanations  do  not  seem  to  connect  with  their  understanding  of  the 
principles  and  concepts  In  the  text. 

Good  students  can  also  accurately  monitor  their  comprehension  failures  and 
successes,  while  studying  examples.  This  Is  consistent  with  what  Is  generally  found  In 
the  literature  (see  Brown,  Bransford,  Ferrara  &  Camplone,  1983).  They  seem  to  detect 
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accurately  when  they  do  and  do  not  understand.  Such  accuracy  In  detection  Is  extremely 
Important  because  It  leads  to  an  attempt  to  try  and  understand  the  material.  Poor 
students,  on  the  other  hand,  seem  less  accurate  at  detecting  comprehension  failures. 
When  they  do,  these  occur  where  mathematical  expressions  are  being  manipulated, 
rather  than  at  places  where  conceptual  principles  are  being  Instantiated. 

Finally,  we  found  that  the  Good  students  used  the  examples  In  a  very  different  way 
from  the  Poor  students.  In  general.  Good  students,  during  problem  solving,  used  the 
examples  for  a  specific  reference,  whereas  Poor  students  reread  them  as  If  to  search  for  a 
solution.  This  Interpretation  Is  consistent  with  the  following  set  of  behaviors.  First, 
Good  students  refer  to  the  examples  less  frequently  than  the  Poor  students,  within  each 
solution  attempt.  Second,  when  they  do  refer  to  the  examples.  Good  students  read  only 
one  or  two  lines,  whereas  Poor  students  read  around  13  lines.  The  Good  students  read 
only  one  or  two  lines  because  they  have  a  specific  goal  In  mind.  Once  they  found  what 
they  needed  from  the  examples,  both  the  Good  and  Poor  students  can  adequately  map 
the  example  and  check  It  for  accuracy.  These  differences  In  the  ways  Good  and  Poor 
students  use  examples  suggest  that  the  Good  (but  cot  the  Poor)  students  have 
understood  the  examples  when  they  studied  It,  as  a  consequence  of  the  explanations  they 
generated,  and  now  can  use  the  examples  as  a  reference. 

Our  results  stand  In  contrast  to  the  mixed  results  often  obtained  In  research 
examining  the  role  of  elaboration  In  problem  solving.  It  has  sometimes  been  found  that 
elaboration  generally  do  not  facilitate  problem  solving  (Reder  et  al,  1986;  Reed  et  al., 
1985).  The  difference  between  these  studies  and  the  present  one  Is  that  we  examine  the 
elaborations  and  explanations  students  spontaneously  produced,  whereas  those  other 
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studies  provided  the  elaborations  for  the  students  to  read.  The  difficulty  of  Interpreting 
results  of  a  study  which  provides  the  elaboration  Is  that  when  It  fails  to  facilitate 
problem  solving,  one  cannot  know  whether  the  elaborations  themselves  are  inadequate, 
or  whether  providing  elaborations  Is  futile. 

Although  the  task  In  which  we  observed  learning  was  how  students  studied 
example-exercises,  we  believe  that  our  results  generalize  to  how  good  and  poor  students 
learn  many  kinds  of  materials,  such  as  declarative  text  (Bransford,  Stein,  Vye,  Franks, 
Auble.  Mezynskl,  &  Perfetto.  1082),  diagrams,  tables,  charts,  and  so  on,  as  well  as 
solving  other  types  of  problems.  In  fact,  our  results  are  reminiscent  of  an  earlier  study  by 
Gagne  and  Smith  (1962)  which  showed  that  asking  students  to  verbalize  while  they  solve 
a  3-dlsc  puzzle-type  problem  correlated  significantly  with  success  at  solving  problems. 
Although  Gagne  and  Smith  did  not  report  Individual  differences,  they  were  able  to 
categorize  the  protocols  Into  4  types: 

1)  those  which  oriented  toward  single  moves  In  the  solution  of  the  problems,  with 
explanations  such  as: 

*on/y  possible  move  * 

’Just  to  try  it ' 

’don ’t  know  ’ 

2)  those  which  anticipated  to  the  extent  of  two  moves,  with  comments  such  as: 

% 

’To  get  at  the  larger  disc  ’ 

’to  free  up  a  space  ’ 


3)  reasons  which  anticipated  sequences  of  moves,  such  as: 
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'move  03  with  a  three-disc  sequence  ' 

*if  disc  is  odd-numbered,  move  to  circle  B’ 

4)  and  reasons  which  explained  the  principles,  such  as: 

'more  odd-numbered  disc  in  the  clockwise  direction  ' 

'move  even-numbered  discs  in  the  counterclockwise  direction*. 

In  light  of  our  present  analysis,  we  would  expect  the  poor  solvers  not  only  to  produce 
fewer  explanations  (l.e.,  less  articulation),  but  also  to  produce  explanations  that  are  not 
necessarily  Justifications  (as  those  in  Type  1).  We  would  expect  good  solvers  to  produce 
both  more  explanations  and  also  to  produce  explanations  which  basically  state  subgoals, 
(as  those  In  Type  2).  which  Induce  a  goal  from  a  sequence  of  actions  (as  those  In  Type  3), 
and  which  relate  actions  to  principles  (as  those  in  Type  4).  Unfortunately,  Gagne  and 
Smith  did  not  break  their  students  down  Into  good  and  poor  solvers.  In  other  words,  we 
do  not  think  that  articulating  an  explanation  per  se  Is  the  critical  factor  (as  suggested  by 
Gagne  and  Smith):  rather,  what  the  students  articulated  Is  the  most  Important  factor. 

Why  do  self-explanations  help  understanding  and  problem  solving?  To  put  it  In 
another  way,  what  Is  learned  from  self-explanations?  We  have  several  tentative 
conjectures,  which  may  not  be  mutually  exclusive.  One  Interpretation  Is  that  self- 
explanations  consist  of  the  creation  of  Inference  rules  that’  are  instantiations  of  the 
principles  and  definitions  Introduced  In  the  text.  These  Inference  rules  are  specific  to  the 
example  presented.  (They  may  be  generalized  later  during  episodes  of  learning  from 
problem  solving.)  For  example,  we  can  collapse  related  explanations  Into  Inference  rules. 
They  are  related  usually  If  they  refer  to  the  same  conditions.  For  example,  the  last  5 
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explanations  shown  In  Fig.  5  (III  -  VII),  can  be  converted  to  three  Inference  rules  (shown 
In  Fig.  7). 

Rule  1:  IF  there  Is  a  body,  and  It  has  weight 

THEN  the  weight  will  act  as  a  force. 

Rule  2:  IF  there  are  forces  upholding  a  body 

THEN  they  are  resistant  to  the  body’s  weight  and 
they  are  equal  to  he  orce  of  weight. 

Rule  3:  IF  (there  Is  the  force  of  the  weight  of  the  body 

and  there  are  resistance  forces...) 

THEN  they  will  all  equal  out. 

Figure  7:  Coding  of  lines  6-i3  of  Figure  5  Into  three  Inference  rules. 

These  Inference  rules  may  be  taken  as  Instantiations  of  the  definition  of  a  force  (Rule  1), 
application  of  Newton's  Third  Law  (Rule  2),  and  Instantiation  of  Newton’s  Second  Law 
(Rule  3).  In  fact,  Larkin  and  Simon  (1987)  found  It  necessary  to  model  problem  solving 
by  the  use  of  similar  kinds  of  Inference  rules.  The  present  study  provides  evidence  to 
support  such  a  claim.  In  fact,  our  data  further  support  Larkin  and  Simon’s  (1987) 
conjecture  that  'students  may  well  be  unable  to  solve  problems  In  part  because  they 
learn  principles,  and  do  not  translate  them  Into  Inference  rules'  (p.  75).  As  roted  earlier, 
we  found  no  differences  In  the  abilities  of  the  Good  and  Poor  students  to  write  down  the 
declarative  definitions  and  principles  Introduced  In  the  text  part  of  the  chapter. 
However,  unless  students  translate  the  declarative  knowledge  Into  specific  procedural 
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Inference  rules,  they  cannot  use  them  to  solve  problems. 

Another  Interpretation  of  self-explanations,  based  on  the  analyses  of  the 
explanations  which  related  to  Newton's  Laws,  Is  the  Idea  that  students,  while  reading  the 
text,  have  only  encoded  a  subset  of  the  components  of  the  Laws.  That  Is,  some  features 
of  the  Laws  are  not  noticed  as  Important,  until  one  sees  that  they  are  emphasized  In  the 
example.  Through  self-explanations,  the  students  are  relating  these  new  components  of 
the  Laws  to  components  they  already  know. 

A  third  way  to  look  at  the  role  of  self-explanations  Is  to  say  that  the  declarative 
Laws  have  been  encoded  entirely,  but  that  some  components  are  more  accessible  than 
others,  and  thus  more  easily  articulated  explicitly  (as  In  the  case  of  answering 
Declarative  questions  about  them).  On  the  other  hand,  self-explanations  during  studying 
examples  may  make  other  latent  or  implicit  components  more  accessible.  Our  data 
cannot  discriminate  between  the  states  of  knowledge  in  which  this  Information  was 
available  but  inaccessible  versus  not  available  at  all. 

Another  possible  mechanism  underlying  self-explanations  may  be  that  self- 
explanations  produce  a  qualitative  constraint  network  which  represents  knowledge  of  the 
solution  steps.  For  Instance,  a  system  of  trigonometry  constraints  could  link  together  a 
general  equation,  such  as  +  F^^^  =  0.  the  diagram  of  forces,  and  a  specific  equation, 

0  o 

-F^cos30  +  Fj^cos-lS  =  0.  When  a  similar  problem  Is  encountered,  say  one  with 
different  angles,  qualitative  propagation  through  the  constraint  network  can  yield  a  plan 
for  the  quantitative  solution.  More  Importantly,  the  constraints  can  propagate 
information  In  both  forwards  and  backwards  directions,  so  that  the  above  constraint 
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system  could  be  used  to  generate  a  plan  for  solving  for  one  angle  given  the  values  of  the 
other  angle  and  the  net  force.  Such  qualitative  constraint  networks  may  be  the  kind  of 
Inference  structure  generated  by  self-explanations,  at  least  Initially  (VanLehn,  personal 
communication). 

Finally,  models  do  exist  In  the  AI  literature  which  can  generalize  from  a  single 
training  example  (Lewis,  1986;  Mitchell,  Keller  &  Kedar-Cabelll,  1986).  Such 
generalizations  are  acquired  by  constructing  a  proof  (explanation)  of  how  an  example  Is 
an  Instance  of  the  concept  which  It  exemplifies,  then,  within  the  formal  theory  of  the 
task  domain,  the  explanation  Is  generalized  to  cover  a  larger  class  of  examples.  Our  data, 
at  a  gross  level,  provide  empirical  evidence  to  suggest  that  such  explanation-based 
learning  may  Indeed  be  the  way  students  generalize  from  single  examples.  However,  we 
need  a  more  detailed  look  at  these  models  to  see  the  extent  to  which  they  can 
accommodate  our  data.  In  Lewis'  (1986)  model  (EXPL),  the  system  basically  contains  a 
set  of  heuristics  for  detecting  the  relation  between  the  action  taken  and  the  outcome  of 
an  action.  For  example,  In  learning  text  editing,  suppose  a  student  Is  observing  a 
demonstration  of  the  following  action  and  Its  outcome:  the  screen  draws  a  box  around 
an  object,  and  shortly  after,  the  object  In  the  box  disappears.  An  IDENTITY  heuristic 
will  be  able  to  conjecture  that  the  relation  between  the  action  of  drawing  a  box  around 
an  object  causes  the  object  to  disappear.  Thus,  the  function  of  explanations  Is  to  apply 
these  heuristics  to  come  up  with  a  hypothesis  of  the  relation  between  the  action  and  the 
outcome.  Notice  that  these  heuristics  are  domain-independent.  In  our  data, 
understanding  an  action  and  Its  outcome  requires  conjecturing  not  only  a  relation 
between  the  action  and  Its  outcome,  but  also  to  explain  how  that  action-outcome  pair  Is 
an  Instantiation  of  a  principle  which  Is  part  of  the  domain  theory.  Thus,  the  greatest 
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difference  between  what  our  students  can  do  and  what  EXPL  can  do  Is  that  EXPL 
cannot  capture  the  role  of  the  domain  knowledge.  In  fact,  the  to-be-learned  domain 
(text-editing)  Is  very  syntactic  In  nature,  and  does  not  require  domain  knowledge  In 
order  to  understand  the  example  actions. 

In  Mitchell  et  al.'s  (1986)  system  (EBS),  however,  domain  knowledge  Is  provided. 
The  contrast  between  Mitchell's  theory  and  our  data  Is  that  Mitchell's  system  has  *100 
much"  domain  knowledge  jri  tbo  following  sense:  when  an  example  has  to  be  proved  to 
be  an  Instance  of  a  goal  concept,  the  system  already  has  a  clear  and  a  complete 
definition  of  this  goal  concept.  Thus,  the  function  of  explanation  Is  to  produce  a  proof 
that  the  example  Is  an  Instance  of  the  goal  concept.  This  may  require  that  some  features 
of  the  example  be  Inferred  so  that  the  features  of  the  example  will  match  those  deaned 
In  the  goal  concept.  If  we  assume  that  the  knowledge  that  permits  these  Inferences  .to  be 
made  on  the  example  concept  Is  analogous  to  the  domain  knowledge  of  physics,  then  this 
particular  aspect  of  the  model  does  seem  to  simulate  parts  of  our  students'  behavior. 
However,  the  key  difference  between  the  knowledge  possessed  by  the  EBS  system  and 
our  students  Is  that  EBS  has  all  the  knowledge  of  the  goal  concept,  whereas  our  students 
do  not.  For  example,  our  students  do  not  have  a  complete  understanding  of  Newton's 
Laws.  Furthermore,  the  function  of  students'  explanations  seems  to  be  both  the 
enhancement  of  their  Initial  understanding  and  generation  of  new  understanding. 

In  sum,  while  our  data  does  seem  to  support  these  two  AI  models  In  a  general  way, 
detailed  analyses  point  out  that  these  models  are  Incomplete  In  describing  actual 
learning,  mainly  because  students'  Initial  learning  from  text  Is  more  limited  than  the 


EBS  model,  for  example,  and  yet  their  subsequent  learning  from  the  examples  Is  more 
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sophisticated. 

We  conciude  by  once  again  quoting  Simon  (1979),  with  regard  to_  the 
characteristics  of  the  learner; 

It  is  left  largely  to  the  student  (by  examining  worked-out  examples,  by  working 
problems,  or  in  some  other  way)  to  generate  the  productions,  the  condition- 
action  pairs,  that  are  required  to  solve  problems  (p.  92,  emphasis  added). 

We  believe  that  the  "some  other  way*  that  students  learn  is  via  generating  and 
completing  explanations.  Hence,  we  have  provided  evidence  for  Simon’s  Insight  that 
•students  learn  both  by  being  taught  and  by  self-instruction*  (p.  87).  whereby  self- 
instruction  is  mediated  by  self-explanations. 
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Notes 

^Newton’s  First  Law  states  that  ■Every  body  persists  In  Its  state  of  rest  or  of 
uniform  motion  in  a  straight  line  unless  it  is  compelled  to  change  that  state  by  forces 
impressed  on  it. 
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